negotiator.qualitymeasures
Class OpponentModelMeasures

java.lang.Object
  negotiator.qualitymeasures.OpponentModelMeasures

public class OpponentModelMeasures
extends java.lang.Object

This class specifies a set of opponent model measures used to measure the performance of an opponent model during a negotiation. Note that the measures are computationally heavy and computed during the negotiation. This entails that it is recommended to use the time-independent rounds protocol or the normal time-based protocol with the pause functionality. This work implement the measures discussed in "Towards a quality assessment method for learning preference profiles in negotiation" by Hindriks et al. Additional measures were added to get a better view of point estimation and distance between sets of points.

Author:

Mark Hendrikx

Constructor Summary

OpponentModelMeasures(UtilitySpace ownSpace, UtilitySpace opponentUS)
Creates the measures object by storing a reference to both utility spaces and calculating the real Kalai bid.

Method Summary

double	calculateAvgDiffBetweenBids(OpponentModel opponentModel)
double	calculateAvgDiffBetweenIssueWeights(OpponentModel opponentModel)
double	calculateAvgDiffParetoBidToEstimate(UtilitySpace estimatedSpace) Calculates the average difference between the real estimated pareto bids and their estimated utility for the opponent.
double	calculateKalaiDiff(BidSpace estimatedBS) Calculates the absolute difference between the estimated Kalai point and the real Kalai point.
double	calculateNashDiff(BidSpace estimatedBS) Calculates the absolute difference between the estimated Nash point and the real Nash point.
double	calculateParetoFrontierDistance(BidSpace estimatedBS) This methods calculates the Pareto frontier distance using the following steps: 1.
double	calculatePearsonCorrelationCoefficientBids(UtilitySpace estimatedSpace) Calculates the Pearson correlation coefficient by comparing the utility of each bid estimated by the real and estimated opponent's utility space.
double	calculatePercCorrectlyEstimatedParetoBids(BidSpace estimatedBS) Calculate the amount of real Pareto bids which have been found by the opponent model.
double	calculatePercIncorrectlyEstimatedParetoBids(BidSpace estimatedBS) Calculate the percentage of bids in the estimated Pareto bids which is really Pareto optimal.
double	calculateRankingDistanceBids(UtilitySpace estimatedSpace) Calculates the ranking distance by comparing the utility of each bid estimated by the real and estimated opponent's utility space.
double	calculateRankingDistanceWeights(OpponentModel opponentModel) Calculates the ranking distance by comparing the utility of each weight estimated by the real and estimated opponent's utility space.
int	getOpponentBidIndex(Bid opponentBid)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OpponentModelMeasures

public OpponentModelMeasures(UtilitySpace ownSpace,
                             UtilitySpace opponentUS)

Creates the measures object by storing a reference to both utility spaces and calculating the real Kalai bid.

Parameters:

ownSpace - utilityspace of self

opponentUS - utilityspace of opponent

Method Detail

calculatePearsonCorrelationCoefficientBids

public double calculatePearsonCorrelationCoefficientBids(UtilitySpace estimatedSpace)

Calculates the Pearson correlation coefficient by comparing the utility of each bid estimated by the real and estimated opponent's utility space. Higher is better.

Parameters:

opponentModel -

Returns:

pearson correlation coefficient

calculateRankingDistanceBids

public double calculateRankingDistanceBids(UtilitySpace estimatedSpace)

Calculates the ranking distance by comparing the utility of each bid estimated by the real and estimated opponent's utility space. Lower is better.

Parameters:

opponentModel -

Returns:

ranking distance

calculateRankingDistanceWeights

public double calculateRankingDistanceWeights(OpponentModel opponentModel)

Calculates the ranking distance by comparing the utility of each weight estimated by the real and estimated opponent's utility space. Lower is better.

Parameters:

opponentModel -

Returns:

ranking distance

calculateKalaiDiff

public double calculateKalaiDiff(BidSpace estimatedBS)

Calculates the absolute difference between the estimated Kalai point and the real Kalai point. Note that we are only interested in the value for the opponent.

Parameters:

opponentModel -

Returns:

difference between real and estimated Kalaipoint

calculateNashDiff

public double calculateNashDiff(BidSpace estimatedBS)

Calculates the absolute difference between the estimated Nash point and the real Nash point. Note that we are only interested in the value for the opponent.

Parameters:

opponentModel -

Returns:

difference between real and estimated Nashpoint

calculateAvgDiffParetoBidToEstimate

public double calculateAvgDiffParetoBidToEstimate(UtilitySpace estimatedSpace)

Calculates the average difference between the real estimated pareto bids and their estimated utility for the opponent.

Parameters:

estimatedSpace -

Returns:

average difference in utility for the Pareto optimal bids

calculatePercCorrectlyEstimatedParetoBids

public double calculatePercCorrectlyEstimatedParetoBids(BidSpace estimatedBS)

Calculate the amount of real Pareto bids which have been found by the opponent model. Note that the estimated utility space may have more or less Pareto bids than there really are.

Parameters:

estimatedBS -

Returns:

percentage of found real Pareto bids

calculatePercIncorrectlyEstimatedParetoBids

public double calculatePercIncorrectlyEstimatedParetoBids(BidSpace estimatedBS)

Calculate the percentage of bids in the estimated Pareto bids which is really Pareto optimal.

Parameters:

estimatedBS -

Returns:

percentage of real Pareto optimal bids given the set of estimated Pareto optimal bids

calculateParetoFrontierDistance

public double calculateParetoFrontierDistance(BidSpace estimatedBS)

This methods calculates the Pareto frontier distance using the following steps: 1. Map the estimated Pareto-bids to the real space. 2. Calculate the surface beneath the real Pareto bids and estimated Pareto bids. 3. Subtract the surfaces and return the absolute difference. Note that the Pareto frontier difference can be positive and negative. In general, the mapped estimate of the Pareto frontier will have less surface; however, it can happen that less Pareto-points were estimated. In this case a Pareto-point is missed, and it can happen that the surface is therefore larger.

Parameters:

estimatedBS -

Returns:

getOpponentBidIndex

public int getOpponentBidIndex(Bid opponentBid)

Parameters:

opponentBid -

Returns:

index of the opponent's bid in the sorted outcome space

calculateAvgDiffBetweenBids

public double calculateAvgDiffBetweenBids(OpponentModel opponentModel)

Parameters:

opponentModel -

Returns:

average difference between the real and estimated utility

calculateAvgDiffBetweenIssueWeights

public double calculateAvgDiffBetweenIssueWeights(OpponentModel opponentModel)

Parameters:

opponentModel -

Returns:

average difference between the real and estimated issue weights